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ABSTBACT 

The evaluation report is one of seven produced for 
the Occupational Exploration Prograi (OEP), a series of simulated 
occupational experiences designed for junior high school students* 
Describing the pilot testing of the siaulation dealing with product 
services, the report contains sections describing the siaulation 
context, evaluation procedures, results, and a Beviser^s Inforaation 
Suaaary (BIS) • In the siaulation dealing with bench electrical 
repair, students had the opportunity to repair defective transistor 
radios, provide supplies to product service personnel, or direct 
other product service workers* Occupational roles included foreaan, 
repairaan, and supplyaan*^ The experiaental design involved tvo 
Colorado schools, vith a^'total of four experiaental and four control 
groups involving 87 eighth and ninth graders* Instraaentation 
included knowledge and affective testing, student and teacher 
guestionnaires, and a panel review* Analysis of variance and other 
descriptive statistics vere eaployed, and reliability estiaates vere 
calculated. Analysis of variance results revealed that the siaulation 
had a positive iapact on both the student ^s occupational knowledge 
and occupational preferences* The BIS records and extrapolates trends 
related to the strengths, weaknesses, and recoaaendations froa all 
data sources* Appended aaterials include the evaluation instruaent 
used and the observer fora* (HS) 
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ABSTR/VCT 
PRODUCT SERVICES 
EVALUftTIOH REPORT FOR THE OCCUPAglOML EXiPLORATION PROGRAM 
5y: James W. Altschiild; Janice Lave; Roger Brmmj Sandra Pritz 



This report is one of. seven evaluation reports produced for the Occupational 
Exploration Program. The Occupational Exploration Program (O.E.P.) is 
funded "by the national Institute of Education and is a joint development 
effort of The Center for Vocational Education (The Ohio State University) 
and the Jefferson County, Colorado public schools. O.E.P. is a series of 
e:q)eriences designed to provide junior high school students \n.th the 
opportimity to explore occupations. One of the najor vehicles for explo- 
ration is the simulation techjiique. In 'FY' 1971^,12 simulations were 
developed and seven of those twelve were pilot tested. This report describes 
the pilot testing of the simulation dealing with Product Services. The 
report contains sections describing sirawlation context, evaluation procedures, 
results and a Reviser's Information Summary (RIS). The RIS is useful for 
a varieiy of purposes and includes the strengths of the simulation as well 
as its weatoiesses. Below is a synopsis of the specific content of the 
report. 

SIMUIATION COSTEXT ; The situation in this simulation deals vdth bench 
electrical repair. Students have the opportunity to repair defective tran- 
sistor radios, to provide supplies to product services personnel, or to 
direct other product services ^rorkers. The occiipational roles included are: 
foreman, repairman, and suppl^rman. EXPERPEHTAL DESIGIT : For evaluating 
this simulation, k schools, two ftom Jefferson County, Colorado and two 
fipom Denver, Colorado were used, each school having one experimental and one 
control group. A teacher facilitated the implementation of the simulation 
with each experimental grotip. The experimental and control groups con- 
sisted of 8th and 9th graders; 39 students in the four experimental groups 
and 1»8 students in the four control groT;?)s . A modified laboratory or 
quasi-experimental setting was utilized for product tryout. PISTRUI.EII- 
TAIIOU" A 33 item multiple choice taiovrledge test, "V/hat Do You Know? , 
and a 6 item affective test, "What Do You Like?" were administered as pre- 
and posttests measuring stxident knowledge gain and attitudinal change. 
The student post module questionnaire, "V^hat Do You Think?", administered 
to the experimental group after completion of the simulation, measured 
student perceptiors of the module. Two teacher questionnaires and one 
panel review were designed to obtain teacher perceptions of the simulation. 
Observers were utilized to collect additional information about module 
lr5)lementation. AITALYSIS : The knowledge test and affective test results 
were derived through analyses of variance. Other descriptive statistics 
were employed where appropriate (i.e., frequency, percentage, percent 
change) . Reliability estimates were calculated to obtain the internal con- 
sistency estimates of the knowledge tests and to determine inter-coder and 
intra-coder for the attitude scale. RESULTS ; The AlIOVA results reveal that 
the similation had a positive impact on 1) student knowledge in the product 



services field (p<.05) and 2) student occupational preference (p<.01). 
This is also corroborated by student and teacher comments collected from 
questionnaire data, Rg/ISOP/S SiFOPuWiTIOn SlvyARY : T.he RIS vas designed 
to not only assist rcvisors to assimilate 5.ni"onnation collected during the 
pilot-test, but also as a unique iray of stwniarizing the data. The summary 
is a record of the strengths, weaknesses and recommendat5.ons for revisers 
from all data sources (i.e., stxident tests, student questionnaires, 
observer forms, teacher questionnaires, etc,)* Trends have been erfcra- 
polated ^7hich list the most apparent strengths and weaknesses as well as 
reconanendations to be considered in the revision of the singulation. 



Actaionrledgement s 



An evaluation report is usually a product of the endeavors of 
many individuals. The authors of this report therefore wish to 
thank: 

1. Patricia Shively for helping in the develppment of all of 
the instrumentation used in the evaluation of this module; 

2. The teachers 9 administrators , and students in Jefferson 
County, Colorado and Denver, Colorado who, by 
participating in the use of educational materials and 
in the^eisting^f those materials, made this evaluation 
report possible; 

3. Jon Schaffarzick, Michael Hock, and David Hanpson of 
the National Institute of Education for their siqjport 
of this effort; and 

k. The twelve project staff menibers identified on the cover, 
who, by their sitpport, expertise and/or direction 
contributed to the production of this report* 



Table of Contents 



PAGE NUMBER 



I. BRIEF DESCRIPTION OF THE MODULE ' 1 

II. DESCRIPTION OF EVALUATION PROCEDURES EMPLOYED 6 

A. Specific Saiaple Used ^ 6 

B. Types of Classes and Groupings 12 

C. Experimental Design as Implemented 13 

D. InGtrumentation-InEtrument Specifics 17 

III. RESULTS 25 

A. Knowledge Test 25 

B. Attitude Scale 29 

C. Student Questionnaire 33 
D* Teacher Midway Questionnaire and 1^3 

General Module Evaluation 

E. Teacher Mid/Post Module Panel Reviews ^5 

F. AnaQysis of Var5ance For Knowledge Test Data ^^9 

G. Analysis of Variance For Attitudinal Data 50 

IV. REVISER'S INFORMATION SUMMARY 51 



APPENDICES 

A. Knowledge Test - "What Do You Know?" 
Attitude Scale - "What Do You Like?" 

B. Student Questionnaire - 'Vhat Do You Think?" 

C. Midway Questionnaire 
General Module Evaluation 

D. Observer Form 



ERLC 



7 



PRODUCT SERVICES MODULE 



Brief Description of the Module 

The purpose of this simulation Is to Introduce students to some of 
the occtipatlons In the product services field. Product services has been 
defined as €m extension of the manufacturing process which provides for 
the installation, maintenance, alternation, and repair of man-made items* 
Speciflcall^r, this module deals with the phase of product services known 
as bench electrical repair. There are nine parts in this simulation: an 
introduction to the simulation > a preview, a preparation, five tasks and 
a summary* 

In the Preview ,* stiidents read the Preview Handbook a nd are introduced 
to the product services' industry. They learn that product services is an 
extension of the manufacturing and marketing process, and that installing, 
maintaining, repairing, and altering are the four types of work within 
product services. The students are told that in this simulation they 
will have the opportunity to either repair transistor radios, provide 
supplies to product services personnel, or direct workers. 

All stixdents who elect to participate in this simulation then proceed 
to the Preparation Phase . Students first read the Preparation Ifendbook 
and learn about the three jobs within the module: repairman, stq>plyman, 
and foreman. They then fill out job applications tising the information 
on the job qualification cards included with the activity and personal 
information from their own lives. 

♦Prior to the preview, the students have seen a slide-tape and/or read a 
booklet entitled Introduction to Simulation. 
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Next, the students view the videotape "Interviewing". Students then 
proceed by giving their ai^plications to the personnel manager of the 
Product Services Company who is played by their teacher, and actively 
interview for the position. After interviewing each applicant, the person- 
nel manager decides who will fill each role. (The teacher is advised to 
choose foremen with leadership abilities since the flow of the simulation is 
dependent upon their competence. In addition, it is suggested that the 
teacher conibine both good readers and non-readers in the repair teams.) 

After students are selected into the roles, they begin the majjor 
tasks of the simulation. The first task is Role Preparation. The students 
prepare to assume their roles by reading the "Job Description" section of 
the Product Service Company Manual . The students become aware of their 
duties in the simulation, the procedures to be employed, and their actml 
job assignment « 

After reading the "Job Description" section of the handbook, the 
students proceed to the next task. Parts Identification. Before they can 
effectively assume their roles, the students need to learn the names and 
uses of the various tools and electrical parts which are included in the 
simulation. The students are teamed in pairs and read the "Pasrts Identi-» 
fication" section of their manual. Working in pairs with flash cards, they 
learn to recognize pcurts and tools by name. 

After students have learned to identify the parts and tools, they 
proceed to the next task. Testing the Multimeter. The students commence 
the task by reading the section "Using the Meter" in the Company Manual s 
In this section, the students learn about the function and operation of a 
multimeter. They are given both voltage test panels from which they 
learn to take voltage measurements and resistor test panels from which 



they learn to take resistance measurements. 

After the students have learned to operate the nultimeter and take 
acciorate measurements, they proceed to the next task, Soldering. The 
students begin this task by reading the ^'Soldering" section in their 
coopany manual. In order to repair electrical devices, the student must 
learn how to make good connections using solder. In a step-vise fashion, 
the st\xdents are given directions on how to solder correctly. After 
coopleting the soldering exercise, stxidents check their work by taking 
multimeter msasxarements . Depending i^n the meter reading, the students 
either correct faulty soldering or proceed to the next task. 

When the correct readings on the resistors are obtained, the students 
proceed to the next task. Troubleshooting, in this task, the students 
simulate the operation of a product services cooq[>any and actxially repair 
defective transistor radios that have been supplied. In this task, the 
foreman is responsible for: 1) correctly tagging the radios, 2) checking 
the work flow of the repair team, 3) answering business letters, k) checking 
repaired radios, 5) computing the cost of each radio repaired using the 
manual •s ''Repair Cost Policy" section, 6) completing the warranty for the 
radio, 7) providing assistance to the repairmen and st^ly personnel and, 
8) returning repaired radios to siqpply for shipping. 

The supply person Is responsible for: l) providing the parts and tools 
needed by the foreman and repairmen, 2) the inventorying of all tools and 
parts, 3) ordering needed parts, k) delivering mail to the foreman, and 
3) coiqpleting the shipping lists and bills for orders. 

The repair person is responsible for: 1) troubleshooting and repairing 
defective transistor radios, 2) completing a repi-ir record for the radio 
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repair task^ and 3) coiqpletiiig parts request forms. When all radios are 
repaired 9 the foreman coiqputes job costs and the st^lynan completes the 
final inventory of tools and parts. When this task of the module is 
completed 9 the student may go on to the sunmiary activity. 

In the Sumnary ^ the students first share their experiences and 
personal feelings about the simulation in a formal meeting vhich is chaired 
by one of the module *s participants. The students and/or teacher has the 
option of inviting another class or group of students to the meeting. The 
purpose of the second part of the sunmary is to acquaint students vith 
the D.O.T. (Dictionary of Occtqpational Titles) since it is perceived 
as a valuable source of career information for students to become acquainted 
with. 
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The foUofffing table presents the estinated time requirenents 
needed for etch activity in the addule* 



TABLE 1 • Estinated Tlise Required for 
Sinulation Coogponents 

ESTIMATED TIME 

activity/task IN CLASS PERIODS* . 



Introduction to Simulation 


1 


Preview 


1 


Preparation 


1-2 


Role Preparation 


^ 1/2 - 1 


Parts Identification 


1/2 - 1 


Using the Meter 


11/2-2 


Soldering 


1 


Troubleshooting 


3 


Sunnary 


3 


Total 


12 1/2 - Ik 1/2 



*A class period is assumed to contain afproximately minutes. 
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II* Daicrlptlon of Evaluation Procedural Bg)loyed 
A. Specific Saaple Ug»d 

!• SchoolB » For this module tvo Jefferson Coontj and two Denver 
schools vere used. In each school there vas one experiaental 
and one control group. The schools and the teachers nere 
selected ria discussion with adainistrators and teachers in 
each of the districts. A brief description^ of the schools 
follom. 

Alaaeda Junior Hig^x School (Ctrades 7*9). Jefferson Countar . 

Alamda Junior Hi^^ School is a ssiall school irith apprcoci- 
aately 700 students in grades seven throu£^ nine. It seesis to 
have a fairly stable school population in that school records 
Indicate that over seventy percent (72^) of the ninth grade 
population have been in this pftrtlcular school for three consecu-* 
tlve years. Additionally, very jTev of the ninth graders have 
attended acre than two elementary schools. Lorge-Oborndike tests 
administered at the school indicate a fairly noraal distribution 
of student ability. The distribution of parental occupations 
shows that k6 percent of the nothers are working and that alaost 
3^ percent of the fathers are in aazui^rial, professional^ or 
skilled positions. The school population is prlnarily Caucasian 
(93 percent) with the remaining seven percent ccmdog trm other 
minority groups. 

^Descriptions were obtained by JcAin Radloff of the Jefferson County 
project staff. 
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Wheat Ridge Jmilor High School (Gradea 7^9) > Jefferion Comnty > 
Wheat Ridge Junior Hi^ School ia a aaall achool with 
approKiaately 725 atudenta in gradea 7-»9* Twsn^ atudenta are 
claaaifiad aa mentally retarded. Ganerally, the achool drawa ita 
atudent hody ttcm a aiddle claaa, blue collar area. About 30 
atudenta ccaie from fanlliea receiving Aid to Dependent Children 
(ADC) and nany atudenta are ttcm divorced honea. The area of 
Jeff Co. repreaented by thia achool baa aany older aingle faadly 
houaea. There la a aizeable retiired aubgroup within the area 
population. The atudenta are priMrlly white ( 9390 vith the reat 
( T%) having Spaniah aumaaea. The achool reporta that atandard^- 
Ized testa results Indicate that school scores are laprorLng and that 
it is either at or above diatrlct noma in noat cases. 
Hanilton Junior High School (Grades 7-9) « Denver . 
Hamilton Junior High School la a large achool with approadmtely 
I96OO students enrolled in gradea aeven throu^^ nine. The area 
served by the school is quite large and over sixty percent of the 
students at Haailton are bussed in each day. The atudenta generally 
cone froa the middle inc(»e range but there are sone students tram 
tipper incone areats. Student achievement seeas to be relatively high. 
(According to the aiasistant principal, over half of the aeventh 
grade atudents maintain a B or hi^er academic average. ) The racial 
make-up of the school is estimated to be 80 percent Caucasian and 
about 20 percent in minority groups. Further specification of the 
population was not available for this school. 
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Lake is a large Denver jtmior high school with well over a 
tho\isand students* Although demographic data was not avai.lable 
at the time of this writing several factors about the school 
are known. First, it has a sizeable percentage of students with 
Spanish surnames « Secondly, in general Lake has a high rate of 
absenteeism. (As soon as additional data becosnes available, it 
will be appended to this report.) 
2. Teachers 

In each of the four schools, one teacher implemented the 
module with the experimental group of students. The following table 
contains a brief description of the experimental grcjtp teachers, 
the number of years of teaching experience, and the method by 
which they participated in the study. 

Table 2 - Description of Experimental Teachers 





Alameda 


Wheat Ridge 


Hamilton 


Lake 


Sex 


Male 


Female 


Male 


Male 


Subject 
Area 

Specialty 


Counselor 


Media 
Specialist 


Socied 
Stiidies 


Audio-Visual 


Had prior 
experience 
with sim- 
ulation 
techniques 


Yes, as 
teacher 


Yes, as a 
participant 


Yes, as 
teacher and 
participant 
(OEP Educa- 
tion Module) 


No 


Number of 
years 
teacliing 
experience 


13 


16 


6 


23 


Participa- 
tion 

selected/or 
volunteered 


Volunteered 


Volunteered 


Volunteered 


Selected 
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3. Stttdents 

In the toUowlng table the number of students participating In the 
e.iy'^riniental and control groups by school and by sex are presented. The 
results show that males were in somewhat greater proportion in the 
experimental group , whereas the female ratio was greater in the control 
grotrp. 



Table 3 - Frequency* of Experimental and Control Participants by School and Sex 



School 


Alameda 


Wheat Ridge 


Hamilton 


Lake 


Total 




Experi- 
mental 


Control 


Experi- 
mental 


Control 


Experi- 
mental 


Control 


Experi- 
mental 


Control 


Ejcperi- 
mental 


Control 


Males 


6 


8 


5 


7 


6 


1 


5 


3 


22 


19 


Females 


2 


5 


5 


5 


2 


17 


3 


2 


12 


29 


Total 


8 


13 


10 


12 


8 


18 


8 


5 


3k 


1*8 



^Frequency is based on participants with cooplete pre«- posttest data* 



The experimental group participants were selected or volunteered ftom 
the following classes: Alameda « eighth grade students chosen randomly 
tram Language Arts Class; Wheat Ridge - eighth grade students volxmteered 
fjrom arty science , and language arts classes; Hamilton - nine grade 
students volunteered from study hall; and Lake. - ninth grade students 
volunteered fjrom a drama class* 

At Alameda, the teacher felt the students were of average ability. 
At Wheat Ridge, the students appeared to be a heterogeneous gcoup with 
student reading ability ranging ftrom high to low. The students were 
highly motivated and had better than average interest. The teacher 
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commented that there was one stiident who was a behavioral problem in the 
group. At Hamilton, the teacher described the class as being one of 
good readers but non-vocal* The teacher ffom Lake Jxinlor High felt the 
students there were generally above average scholastically and able to 
handle the vocabulary in the module. 

The method by which the control groups were obtained is not totally 
cleart In the testing of h modules in the Spring of I97U it was not 
feasible for one individual to administer the tests. Therefore in each 
school either the experimental group teacher or another educator selected 
and administered the tests to a control groirp. It was suggested that 
testers try to select or sample students similar to those in the experi-* 
mental group, i.e., if the e3q)erimental groi?> was an English class then the 
tester was instructed to obtain a second English class for the control 
group. It is assumed that, to the extent possible, these directions were 
carried out. (Referring to Table 3 9 this may not have been possible at 
Hamilton since a significantly larger proportion of females were selected.) 

In summary, the sampling was far from ideal. It was Impossible to 
conduct more systematic sampling due to program and organizational con«- 
straint within buildings. It should be noted that experimental restxlts 
are based only on students who completed both the pre- and posttest. There 
was sample loss in the testing of the module as described in the following 
table. 

*The time and monetary allocations for the pilot test precluded the use 
of extensive checks on the sampling procediu'es in the field. 
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Table h - Frequency and Percent of Sample Loss by Group 





Original Total 


Sample Loss 


Percent of Total 


Experimental 


35 


1 




Control 


69 


21 




Total 


loU 


22 


21. if. 



Sample loss is aLLways difficult to account for in an experimental 
situation. Some students may have been sick or otherwise out of the class- 
room during the pre- or postter,ilng time. Some students may simply have 
avoided taking the test. 

The sample lo3s in the e^qperimental groiqp is relatively small. However , 
the saosple loss within the control group is considerably higher and far frcm 
ideal. When cociparing the n*s of both groups , the high sanple loss within 
the control group loses some significance since the number of s\ibjects in 
the control group is greater than those in the experimental gror^ (see 
Table k). After considering all factors , it is sxaggested that the sample 
loss will not affect the validity of the results. Therefore, efforts will 
not be made to study it in detail. 
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II. B, Typea of Claises or Qrouplnga 

Knowledge of the type of class oar group setting in which the nodule has 
been tried is laqportant information in regard to interpreting the iBOdule 
results « For the k nodules tested in the Spring of l9Tk a modified laboratory 
setting was utilized. Either a clasarooa or a space within a library was set 
aside for use by atudents p^icipating in the module* When necessary, special 
equipment (e.g., video tape machines, sound on slide projectors, etc.,) was 
provided and if possible, stored in the space designated for the project. It 
was felt that this specialized aarea would tend to: 

- reduce the nuaiber of coopeting or distracting factors for the simulating 
group; 

• be representative of one way in which a school could inplcnent the 
OEP program; 

«- reduce the necessity to move equipment around from period to period; 
and 

- provide a place for students to store materials from one simulation 
day to the next. 

All experimental groups were conducted in this specialized or quasi- 
experimental type of setting. Testing was also generally carried on 
within this setting. 
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II. C. Experimental Design as Implemented . 

There were two constraints surrounding the iinplementatlon of 
the design as specified In the proposal for this module. Given the 
relatively small sample size a decision was made not to Include sex 
as a variable. This eliminates the possibility of studying the test 
scores of males and females, but incl\ision of this variable woiald so 
reduce the cell size as to make meaningful interpretations difficult 
at best. The second constraint concerns the way in which the field 
test design was inclement ed. Four schools were tested, two classrooms 
or groiqps per school. Within each school one group served as the 
experimental treatment and the other as the control treatment. It is 
apparent that no estimates of between class variability can be con^juted 
and that any unique classroom effects are confounded with treatment 
effects . Howe\^3r, the test of treatments and associated interactions 
is assumed to valid.* The design is depicted schematically in 
Figure 1 on the following page. 



*Thi8 statement is based on the presimption that th'^re were no 
unique classroom effects, one that is sigrported in earlier field 
trials. The reader should note that this design was implemented only 
after consideration was given to the practical a8i>ects of implementing 
the design. It was the most feasible one given the field situation. 
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Figure 1 - Scheaatic of the Experimental Design 
for the Product Services Module 



Pretest 



Alameda 
(Jefferson 
County) 



Wheat Ridge 
(Jefferson 

County) 



Experimental 



Posttest 



Control 



Experimental 



Control 



Sl 



Experimental 



Hamilton 
(Denver) 



Control 



Experimental 



Lake 
(Denver) 



Control 



*In order for a student *8 scores to be included in the analysis ^ he 
would have had to participate in both the pre and posttest. 



The analysis will be the same as designated in the project 
proposal for the ^eupational Exploration Program vlth the 

exception that the sex variable has been deleted and two schools were 
added. Of key interest irlU be the interaction betimen the e3qperiMntal« 
control variable (B) and the pre« cmd posttest variable (C). If the 
nodule has had an lq;>act upon students, a significant interaction irould 
be esqpected with the source of the interaction being a siseable 
experimental group gain on the posttest. Separate analyses will be run 
for the total cognitive test scores as iiell as for one dixaension of the 
attitudinal scale. The analyses vill be in accordance vith the 
abbreviated suammry table shoim on the foUoiring page. 
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Table 5 ^ Partial' Anova Suaaary Table 

for the Product Services Module 



Source 


df 


' Potential F Test 


Between Students 


abn>l 




Term No. 

1 A 

2 B 

3 AB 

k d/ab 


a-1 

b-1 

(a-1) (b-1) 
ab(n-l) 


lA 
2/k 
3A 


Within Students 


abn(e*l) 




5 C 

6 AC 

7 BC 

8 ABC 

9 cd/ab 


c-1 

(a-l)(c-l) 
(b-1) (c-1) 
(a.l)(b-l)(c-l) 
ab(c-l)(n-l) 


6/9 
8/9 


TOTAL 


abcn-1 




The independent variables for this module are 


as foUoirs: 


Variable 


Description 




A 


Schools (Alaaeda, Wheat 
- Ridge, Hamilton and Lake) 


Fixed; betmen S's 


B 


Treatment (experimental 
vs. control) 


Fi::ed; between S*8 


C 


Testing (pre. vs. post) 


Fixed; within 

S's (repeated measure) 


D 


Students 


Random; nested within 
AB ccMbinations 
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II. D. Instnonentatlon * Instrmnent Specifics 

!• Knowledge Test ^ What Do You Know? (The test Is appended to 
thl3 report) 

The knowledge test for the Product Services Simulation 
consists of 33 nultlple choice questions. Most of the questions 
contain k response options each. All questions receive eq\ial 
weighting • 1 towards the total if an item was answered correctly 
and 0 if answered Incorrectly or if no response was given • 
setting the range of test scores from 0 to 33, 

Generally 9 the questions are at a low conqprehension level 
with respect to the Bloom Taxon<»iQr« Three basic dimensions are 
emphasized in the test*skllls/requirements, responsibility^ and 
process. An exaople of a skills/requirements question is as 
follows: 

Test Question #31 

Otto Sight is an outgoing person who has spent three years 
in the service doing repair work. What type of product 
service work would best suit him in civilian life? 

a. Working in a product services compatqr repsdring products • 
Installing and maintaining products in customer's homes* 

c. Keeping track of sxxpplies for a product services coorpany. 

d. Making diagrams of products for use in repair work. 

Skills/Requirements qixestlons generally deal with the basic 
abilities or attributes required to function properly in various 
roles or Jobs within a product service company. 

Responsibility questions , on the other hand^ generally deal 
with which individtxal or group has the responsibility foi* getting 
a certain Job done or who has responsibility for making decisions 
at a certain point in tlme^ etc. 



^Indicatfts correct answer 
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An example of a responsibility question is given belov* 
Test Question #27 

Which of the following duties Is the responsibility of 
supply personnel in a product services conpany? 

a. Determining parts needed for repairing a product 
*b, Codxpleting shipping lists for parts 

c. Determining costs for repairing a product based on 
parts* used 

d. Keeping track of what parts were used to repair a 

product 

The third type of questions on the Knowl<;dge Test regards 
the process dimension* Process questions concern understanding 
. the nature of steps involved in doing work such as operating a 
product services company^ understanding what information might 
be necessary to perform a certain function, etc. The following 
is an example of a process question. 
Test Question #l6 

The first step in servicing a product is 

a« Locating the prnhlem(s) in the product 
«b. Labeling the product with the owner *s name 

c. Cleaning the product 

d. Repairing the product (including parts replacement) 
The following table shows the breakdown of test items by test 

content and by the process , responsibility and skills/requirement 
dimensions. The content has been subdivided into two areas: general 
considerations and specific occupations* The test was designed to cover 
most of the major aspects of the content present in the module. 



♦Indicates correct answer 
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Tmble 6 - Analysis of Test Content 



Content Area 


Process 


Responsibility 


Skills, 
ReQuirements 


TotSLl 


General 


15 






15 


Considerations 




















Occupations 














k 




k 


Repairman 


' 3 


X 


5 


9 


Suppl^man 




2 


3 


5 


Total 


18 


7 


6 


33 
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2) Affective Test > What Do Yoa Like ? (The test is appended to this 

report ) • 

The affective test mta designed to sieasure student attltudlnal 
change. The first six mestions asked the student if he/she vould like 
to try doing an activity. The student could respond in one of four ways to 
the item: 

1) Yes, I wiild like to try this 

2) No, I would not like to try this 

3) I*m uncertain about trying this 

k) I don^t have enough infozaation to know if 1 would like to try 
this. 

The scale is scored so that the stronger the preference for trying to 
do an activity, the higher the score. Thus, yes and no responses receive 
the same scale value of 3* uncertain responses receive a 2, and not enough 
information types of responses receive a value of 1. These values are then 
suiODed and used in the analysis of variance described earlier. Summed 
scores can vary f^cm zero (no response whatsoever) to 18. Note the scale 
is scored so that strength of preference, rather than direction of preference 
is the important factor (i.e., yes and no responses, while being in opposite 
directions, represent the same strength of preference and therefore receive 
the same score). 

In addition to the scaled responses, students were encouraged to state 
reeisons for their preferences. These reasons were classified and, in . 
conjunction with the scatled responses, ^fere coded and transferred to machine 
scormble forms. Inter-rater and intra-rater agreement checks were made on 
the scoring x^rocess (See aresults section). The last question of the "What 
Do You Like?'' test asked the student to imagine himself /herself as 
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a Product Services Employee and to give adrlce to another perscxi l>y 
Indleatins irtiat kind of experiences or activities migbt belp him/h^r 
' prepare for a Job In the Product Services Industry (This que8tl<Hi iras 
used on a preliadnary trial basis* The open-ended responses vere 
classified snd coded, but vill not be reported in this docuMnt#) 

3) student Ftost Module Questionnaire • What Do Yon Think? (The Question* 
naire is appended to this report;* 

This questionnaire vas acbninistered to students in the ezperiaental 
group after their cooqpletion of the aodule and its posttest* The 
questionnaire ims designed to measure student perceptions of the nodule* 

The first tven^ questions on this questionnaire vere forced choice 
in nature the student could either agree or disagree vith the statement 
posed in the stem* The twenty questions covered the folloiring k areas: 

• perceptions of specific module parts (questions 1*7); 

• general understaading and ability to follow directions (questions 

8-13); 

• implementation or pacing of the SK)dule (questions lk^l£)i and 
- perceptions about learning (questions 17-20)* 

Besides the first twenty questions there were twelve additional 
questions* Three of these were "check" questions designed to provide some 
probable Indication of scale reliability* The rest of the questions were 
open-ended and asked the student to stqaply short answers or reccamsendations 
for inproving the simOation* Exajnples of areas covered by these questions 
Include: role(s) played; things liked most about the simttlation; things 
liked least about the similatloa; new Interests discovered throu^ the 
simalation, etc* These questions will be sunmariMd and Included on the 
Reviser's Information Susnary (RIS)* 
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Teacfatr Quastionnalres (The qTiestloonairea are appended to this report )« 
Baaically^ two questloonairea irere used for t eating thia module. 
The firsts th e Midway Qgeatl<»nalre > vaa coaqpleted by teachera approsl- 
mately half-way throu|^ the module. Thia qrieationnaire iraa filled out 
just prior to a nid^nodule panel reriev of the fir at half of the nodule. 
The questionnaire ^sdeaigned to cover the initial elenenta of the aixaola* 
tion^ i.e«i the Introduction to Siaulation materials ^ the KrerieVy the 
Freparatlcn Ihaae^ the firat initial taaka^ and the teacher^a overall 
perceptiona up to the nidmy point. The queationa deed irith concerns 
about technical quality^ fit or integration with other aectiona of the 
ttodule^ apprppriateneaa of recooBiended tine allotMutSy problema encountered^ 
recGBBnendationa for change , etc. The queationa ifere prijpuuri3y on a five 
point acale with space for open-ended coaoawnta frequently provided* 

At the end of the module and prior to the poat nodule panel revieir, 
teachera conpleted the General Module Evaluation . Thia queationnaire 
was ainilar to the Midway Queationnaire, except that ita content pertained 
to the last taaks and sumaary phase of the nodule and to the teacher* a 
perceptiona acrosa the entire nodule. It alao contained queationa deaUng 
with atudent and teacher background* Generally, it waa adniniatered at the 
poat nodule panel review seaaion. The questionnaire would require ibout 
2^-30 ninutea to coatplete. 

In conjunction irith the two queationnalrea juat deacribed, two optional 
fona were provided to teachera. Theae were the Media Checkliat and the 
Daily Inventory of Percpetiona (DIP). The Checkliat waa ainply a form that 
teachera could uae if they ao deaired to record their feellnga about media 
used in the sianilation. The DIP was an open-ended diary form available 
for those teachera who were willing (or wanted) to keep daily notea about 
the simalation. 
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Data from the two questionnaires and the optional forms, if completed, 
will be summarized and reported in the Reviser's Information Summary, 

5) Teacher Module Panel Review 

As suggested above, teachers who participated in the pilot test and 
taught the module were convened for one post-module panel review. For each 
section of the module, the reviewers were asked to denote the strengths 
and weaknesses, the classroom solutions applied to overcome weaknesses, and 
recommendations for revision. The main reasons for the panel were as 
follows : 

the panels were a means of obtaining fresher or more recent 
teacher observations; 

- two shorter panels rather than a longer, more tedious panel 
would tend to reduce teacher fatigue; 

- the panels decreased the need for longer questionnaires. 

As in the case of the first three modules tested in the Fall of 1973 > 
panel reviews were conducted in accordance with the panel review guidelines 
generated for the nationwide CCEM project. A meitiber of the panel kept 
detailed notes and after the panel prepared written panel review reports. 
These are included in this document and will be sumnarized on the RIS. 

6) Observer Form (The form is appended to this report.) 

For this pilot test observers were utilized to collect additional 
Information about module iinplementation. Observer data was collected for 
all schools with the exception of Lake Junior High School. At Alameda and 
Hamilton Junior High Schools only one observer for the entire module was 
needed. At Wheat Ridge Junior High School it was necessary to employ two 
observers. The four observers were women, two of whom had college degrees 
and two of whom had high school diplomas . The background and ages of the 
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observers varied considerably. The forms the observers used were a mixture 
of checklist and open-ended formats. Three basic areas were covered; 
media; general ccnmnents; and interaction and activities. The observations 
that were made were reviewed, c Dilated, and sumnarized for the RIS. 
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C. ?• Student Questionnaire : Collated Open-Ended Responses 

to Questions froo the "What Do You Think?" questionnaire 



Question #25 

Name seme of the things you liked most and least about the role(s). 



Liked Most 



Liked Least 



Repairing the radios. 
Working with other people. 
Working wit'i radios 
Soldering 

Leeiming something new 

Answering lexters 

Everything 

Learning radio parts 

Working with meters 

Freedom 

The experience, work 
responsibility 
Taking inventory 
Doing things 



•Reading too many booklets 
-Directions were too coinplicated, flow 

chart too difficult to understand 
-Not long enough to learn enough, too 

short 

-Waiting for the resistor 

-The radio's didn't work 

-VJriting the letters 

-The radios were too simple to fix. 

-Didn't have enough parts 

-Filling out forms, tests 

-Not enough to do 

-Not getting paid 



Question #28 

Name sane of the materials you liked most and least. 



Liked Host 

Video tape "Interviewing" 
Films 

Slide/tapes 
Radios 

Resource materials 
Soldering Iron 
Meter 



Liked least 

-Booklets, reading 
-Filling out forms 
-Slides 

-Not enough parts 
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Question ^5^31 

Naoae some of the things you liked siost and least about the slmlation. 



Uked Most 



Liked Least 



Repairing radios 
Soldering 

Working with tools 
Having good equipment 
Having the work experience 
Working with people In class 
Interesting and exciting 



•Reading booklets 
-Filling o^it forms, tests 
•Lack of all parts needed to fix radios 
•Too short, not enough time 
•The directions & organization of 
simulation was poorly put together 



Question #32 

Student recommendations to inrprove simulation: 

1. Improve the directions and 8iB5)lify the flow chart to repair the radios. 

2. Improve sequencing of materials, number pages of booklets. 

3. Improve directions on how to \ise equipojent. 
U. Allow more time to complete the simulation. 

5. Eliminate some of the reading. 

6. Get better parts, radios and equipment to work wl.th. Get different 
wijre than copper. Some of the equipment didn't work. Need better 
aerial. 

7. First, send radios to school in working condition. Have teacher 
check to see if it worked, then break It down. Have students first 
repair simple radio problems then more complicated radios. 

8. Include more background In basic electronics. 
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III. RESULTS 



2* Post Module Panel Review 



Title of Module ; Product Services 

L.EA. : Jefferson County and Denver County, Colorado 

Panel Leader ; John Radloff 

Panelists ; Jerry Forkner, Hamilton Jr. 

Dorothy lyons, Wheat Ridge Jr. 
Dee Dickson, Alameda Jr* 
Tony Petrun, I^Ve Jr. 

Observer Participants : None 

Date(s) Panel Met ; U/25M 

Number of Hours: 2 
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rv. Reviser's Information Sunanary (RIS) 
A> Description of the Smgnary 

The Reviser's Information Siminary was developed for the purpose of 
assisting revisers to assimilate information collected during the pilot 
test of a module. To accomplish this, information from each source 
available was first reviewed and then only major thrusts or ideas ftrom 
the source were summarized. (These key thrusts or ideas were determined 
by the judgment of the authors of this evalxiation report. ) The summary 
was then transferred to the appropriate location on the large sheets 
which constitute the RIS. Lastly, each column was studied and trends 
were drawn and so recorded at the bottom of the sheet. In ascertaining 
the trends the authors used their familiarity with data, the module, and 
the data collected. 

In general there will be one Reviser's Information Summary sheet per 
part of the module and one-two sheets covering the overetll nature of the 
module* On sheets which pertain to module parts, only some of the data 
sources provided information pertinent to that part. Hence, the sheets 
do have some blanks or missing data cells. The revisers should exercise 
extreme care in interpreting the information on the sheets and should 
always keep in mind that comments on the sheets represent only a summary 
of key points. In addition, it sometimes was most difficult to determine 
a trend in the information obtained. 

B. Use of the RIS 
One way the reviser might use the RIS is as follows: 

1. Read the module ~ become thoroughly familiar with it; 

2. Read the first part of this report (Section I and II) thoroughtly. 
Skim the results compiled in tables (Section III, parts A,B,C,D, 
and E. ) Read section E-2 and the teacher panel review report 
closely; 



Read and study the Reviser's Information Summary, (Consult 
original data sources , if necessary,); and 

Generate a set of revision specifications based upon knoirledge 
of the module, the Reviser's Information Sunoary, project 
developmental criteria and other information, if appropriate* 
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C. REVISER'S INFORMATION 
SUMMARY 
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DATA 
SOURCE 


3T?engt:[S 


rroduct Ser-y 
Overall C02 




STUDEOT 
TSSTS 


The experimental group of students cained 2 points on the 
knowledge test from pre- to postteating* However, due to 
initial pretest mean differences between the control and 
experimental groups the results should be interpreted with 
caution* (See Tables A*2. and F.l.) The ANOVA results from 
the attitude scale reveal that the experimental group's 
strength of preference toward related jobs /skills did change 
significantly (see Table G.l#) 


\ 




< 

JTUDENT 

iUESTION- 

lAIBES 


Students commented that the following things were what they 
liked most about the siraula^tion: 

- repairing radios 

- soldering 

- working with tools 

- having good equipment 

- having the work experience 

- working with people in class. 

Generally, students had positive feelings toward the 
following statements: 

- The booklets and resource materials were easy to 
read {&!%). 

- I learned about how to work with people {m%) ^ 

- I enjoyed working with other students during the 
simulation (8l^). 

- I learned quite a bit about jobs in this field of 
work (75^&). 

- There were not too many forms to fill out with this 
simulation (72^). 

- The pretest and posttest were not to difficult for 
me (75%). 

- The preview, activities and summary fit well 
together (5^%). 


Students commented that| 
what they liked least at 

- reading booklets 

- filling out formi 

- lack of all parti 

- too short, not et 

- the directions &i 
was poorly put t< 

Generally, students had 
following statements: 

- The directions ii 
me (yes 50^, no ; 

- The simulation wi 

- Sometimes, I had 
no kli). 

- Sometimes, I had 
no 66fo). 

- The simulation d: 
the questions I ] 
no hli)^ 




TEACHER 
QUESTION- 
NAIRES 

ERIC 


The teachers were inconsistent in evaluating the overall 
quality of the module. Their ratings ranged fl-om "very 
good" to "very poor". All teachers indicated they would 
use this module again; however, two would use it with minor 
modifications and two would use it after major modifications 
were made. Three of the four teachers said they would 
recommend the simulation to other teachers* However, one 
specifically felt only to teachers with electronics back- 
ground and another felt only after major changes were made 
in the module. The teachers were in greater agreement 
concerning that the main ideas and themes were presented 
with logical consistency within the content of the module. 
The teachers were inconsistent in indicating which materials 
in the module they felt were best. One teacher stated each 
of the following: letter writing, cost policy, tools; the 
interview, radio repair; written materials; and all 
"hands on" materials. All teachers felt the directions in 
the module ver^ clear enough for the students to understand. 
They felt that at least some of the vocabulary was consistent 
with the maturation level of the students. Three teachers 
felt that most of the time the students were able to under- 
stand the concepts presented in the materials and their 
interests were stimulated by the materials. The teachers 

(Continued on the following page) 
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The teachers stated the 
worst: radios (n=2) sine 
slide presentation on c€ 
als without illustratic 
there were errors in the 
of the booklets were not 
A teacher using this sun 
experience in electricil 
teacher felt the worst n 
preview booklet. Three 
breaks in flow of the ac 
culty in implementing tl 
to repair many of the rt 
changed during the simu] 
repair the radios, it c< 


4 



2rvices .vo.lule: 
JonGidsrationc (] ) 



5h 



9 



it the following things were 

about the simulation: 

s 

ms, tests 

ts needed to fix radios 
enough time 

& organization of simulation 
together 

d mixed feelings toward the 

in the materials were clear to 
50^). 

was too short (yes 50^, no h7%). 
d nothing to do (yes 56^j 

d too much to do (yes 3^-^, 

did not help to answer some of 
have about jobs (yes 56^, 



2. 

3. 
h. 
5. 



Student recommendations to improve simulation: 

1, Improve the directions and simplify the flow chart 
to repair the radios. 

Improve sequencing of materials, number pages of 
booklets. 

Improve directions on how to use equipment. 
Allow more time to complete the simulation. 
Eliminate some of the reading. 

6. Get better parts, radios and equipment to work with, 
Get different wire than copper. Some of the equip- 
ment didn't work. Need better aerial. 

7. First, send radios to school in working condition. 
Have teacher check to see if it worked, then 
break it down. Have students first repair simple 
radio problems then more complicated radios. 

8. Include more background in basic electronics. 



following materials as being 
ice they were irrepairable, 
jareers, and the reading mater i- 
ons. The teachers indicated 
repair flow chart. The pages 
►t numbered and out of sequence, 
imulation needs to have some 
ty and/or electronics. One 
material developed vras the 
teachers indicated having 
LCtivity. Students had diffi- 
he task since they were unable 
adios. The students* riotivation 
lation. After being unable to 
ecre':3a' considerably. 



1. Revise equipment (radios) sent to school. 

2. Change student schedules so they meet as a group to 
work on the simulation straight through the day 

a^ opposed to an hour daily for three weeks. 

3. Have module taught by a professional career 
education staff member as opposed to Language 
Arts faculty member. 

^. Career education should be available to students 

for greater period of time. 
5# One teacher felt the module should be taught by 

teachers vith an electronics background. 
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DATA 
SOURCE 



STREiriTHS 



were inconsistent in evaluating student problems with the 
module reading level. Two teachers indicated their 
students had no problems with the reading levels while 
two other teachers felt their students had some problems. 
Three teachers indicated they spent little time reviewing 
the basic concepts presented in the simulation. One teacher 
felt the best material developed for the simulation was the 
video-tape interviewing. The students were receptive to 
to both the simulation as a way of learning and the content 
of the module. Two teachers felt the module helped build 
the students* ability to make career decisions. 



Product 3t 
Overall 

VJEJ 



1. Given student test scores and student comments regarding 
the module, the statement could be made that it was 
somewhat successful. Student test scores are also indi- 
cative of some of the positive impact of the module. 

2. Although teachers were divided with regard to the 
amount of revision necessary, all teachers indicated 
they would use the module agetin. 

3. In general, the module was considered to be logically 
consistent, i.e., the parts seemed to fit together. 

h. The vocabulary in the module was seen as being approxi- 
mately appropriate for the age level, however there was 
some inconsistency of opinion with regard to reading 
level. 

5. Students generally were receptive to both the content 
of this module and the concept of simulation. 



1. A major set of weak 
in the radios conta; 

2. There were major fli 
were problems with 
These two factors w 
some breaks in the 

3. There was some feel: 
benefitted from hav 
start of this modul 

h. Student motivation 
simulation, reaching 
tion phase. It dro 
had difficulty repa 
serious lack of mot 
Summary Phase. 
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I Services vodule: 
X Considerations 



T^r:co:4:G:n)ATio?is for revision 



tknesses relates to technical problems 

iained in the module* 

!laws in the flow charts and thare 

I the directions in the module ♦ 

when combined may have resulted in 

t flow of action* 

sling that teachers would have 

Lving some experience, prior to the 

lie, with electronics* 

I changed during the course of this 

ng a high point during the prepara- 

ropped off for those students who 

miring the radios* There was a 

)tivation noted by teachers in the 



To a high degree the student and teacher recommen- 
dations for changing this module are a direct 
outgrowth or result of the problems- they encountered 
in the pilot test* This is not only true of the 
overall considerations but also of the specific 
parts of the Reviser's Information Summary* 
Therefore it is simply suggested that the re- 
viser read the recommendations stated above and 
carefully review the problems (and recommendations) 
on each specific sheet of this section of the 
evaluation report* 
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Introduction to SinM 



Prom an incremental test* done in the Fall of 
1973 the following results were obtained : &7\ 
(n=15) or more oi the students using the materials 
felt that they understood the materials and that 
the vocabulary was easy to unaerstand. 



V/hen students were questioned \ 
enjoyment of the introduction, 
etc., the picture became soraewl 
* Only of the students 
of enjoying the booklet c 
- Only about l/3 of the st\ 
in terms of liking the iJ 



«Test data was collected from students in Upper Arlington, Ohio 



The slides were generally rated as being "good" by 
3 of the k teachers. One teacher felt they were 
"very good". In addition, the booklet was 
generaJJy rated "good" (n=l) (very good, n=2; 
average n=l). One teacher felt it was good that 
the slide showed a woman executive and co-pilot . 



The synchronized slide advance m 
booklets were too wordy; need t< 
listings. The teachers were in< 
the order of use of the slides 
felt the booklets could be elim: 
information. Two other teacheri 
but in any order; however, insu: 
interest. 



1^ Generally speaking, the Introduction to 
simulation materials were well received. 
This is similar to comments collected frorr* 
other modules. 

2. From the incremental testing in Columbus, 
there were indications that the students 
were understanding the concept presented in 
the materials. 



1. As indicated in other modiil* 
regarding the student inter j 

2. Also, there were some quest] 
illustrations, booklets beij 
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•fith rcjgard to their overall 
I, the quality of the materials, 
twhat more nixed in nature • 
;s were firm in their statement 

or the slides • 
tudents were strongly positive 
illustrations* 


nightly over one-half of the students recommended that 
the slides and booklet be used together, with the slides 
coming first* 


was not working properly* The 
to substitute paragraphs with 
^consistent in recommending 
\ and booklet* Two teachers 
.minated since they duplicate 
trs felt you should use both 
iuring sustained student 


The teachers were inconsistent in their recommendation of 
what order to use the slides ani booklet in the simulation* 
Three teachers felt that both should be used while one 
teacher felt only the slides should be presented* 




• 






lies, there were some comments 
trest factor* 

itions about the quality of 
dng too v/ordy, etc* 

ERLC 


1* Pased upon the incremental test in Columbus, results 
from other modules, and from this module, the 
following recommendations are suggested: 

- revise the introduction with the view of 
increasing student motivational aispects* 

- the first suggestion might be accomplished by 
decreasing some of the wordiness and improving 
illustrations * 
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DATA 
SOtJRCB 


STRENGTHS 


•a?JAKIESSES 


STUDENT 
TESTS 






STUDENT 
QUESTION- 
NAIRES 


Sixty-nine percent of the students felt the 
preview helped to prepare them for the 
simulation* 




TEACHER 
QUESTION- 
NAIRES 


The preview consisted of sound slides and 
student booklet. All teachers rated the tech- 
nical quality for media and/or illustrations 
for booklets as being high. Three teachers 
felt the preview provided students with 
"rather pertinent" information in making 
decisions about module participation. 


The preview slide /tape plus booklets ^ 
students to a "medium" extent. One te 
that the motivating incentives were: t 
and parts recognition. 


TEACHER 
PANEL 




- 


OBSERVER 
FORMS 










TRENDS 


The module preview was received well by 
both teachers and students. The teachers 
not only rated the preview high but in 
additional provided pertinent information 
for students. 


As indicated in the column above, the 
been as highly motivating as one woulc 
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were felt to motivate the 
teacher, however, corptiented 
; the tool chest, equipment, 


Could mediate, develop transparencies on the divisions of 
product services and the mechaniccd and electrical system's 
parts. The D.O.T. should be included within the preview 
phase of the module as opposed to the summary. 


















e materials may not have 
Id hope for. 

ERJ.C 


1. As suggested by teachers, one way of presenting 
the materials could be by developing transparencies 
of product service's divisions, etc. 

2. An important point here, is that the D.O.T. could 

be included in the Preview* The D.O.T. was originally 
placed in the Summary, but as will be noted later, 
the teachers felt it v/ould be more meaningful in the 
Preview Section. 
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DATA 
SOURCE 



STRENGT'S 



STUDENT 
TESTS 



STUDENT 

QUESTIOn- 

KAIRES 



TEACHER 
QUESTION- 
NAIRES 



rEACHER 
■^ANEL 



The majority of the students {12%) were able 
to select a role by themselves* 



Three teachers rated the quality of the video- 
tape as being "high". In addition, they felt 
the preparation section fit well with the 
preview. They felt the initial role descrip- 
tions provided students with adequate and 
ample information from which to select roles • 
Generally, the teachers felt the students were 
able to use the schematic devices for role 
selection with little help. In 3 out of U 
classes, the students were able to independent- 
ly select roles with little difficulty. The 
teacher interview and the flash card method 
for parts identification was generally 
considered to be good by the teachers. One 
teacher sent individual letters to each 
student notifying him/her about their position 



Bighty-one percent of the studen 
gave them little infornation hel 



In some classes, all students wa 
teachers had to draw lots for th 
felt the success of the module d 
the supply man (men). This stat 
teacher's guide, in which the su 
depend upon the leadership abili 



Two teachers felt the intearviswing for job 
was a high point of the simulation and the 
smoothest part of the module. 



8SERVER 



The interviews went well. In one class, the 
teacher sent a letter to each job applicant 
informing them of the job they were "hired" 
for and its requirements. This method led to 
smooth beginning for the pupils to assume 
role playing. 



Students needed a lot of help in 
Spmo students felt they could hav 
\ ; -'.c so much reading. One cla 
wo at their o\m pace and not ha 
to other schools using program* 



TRENDS 




1» There is high agreement across students, 
observers, and teachers concerning the 
success of this activity* In fact some 
teachers considered it to be the high 
point of the module # 

2. Interviewing and the flash card method 
for parts identification were succesGful 
activities in this phase # 

3» The preview and preparation section fit 
well together. 



1. /Mle the teachers felt adequ 
given to students, 8l';o of the 
that » 

2. In some classes, there were p 
students wantin;^ in cer 

2# Students nocdad ir.uch help in 
Tliere vas fcolin-. by the i;tud 
prepared for tV.^i intervi'^v; wi 
\n one cIcLss, there '/:as come 
'/as the learlership role. 
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s feit the role descriptions 
fu] in choosing a role. 




ted to be repairman. The 
other roles 0 One teacher 
pended upon the ability of 
ment conflicts vrLth the 
cess of the module is said to 
y of the foreman. 












illing out job applications, 
prepared for the interviev; 
s felt they should be able to 
e to mai itain pace in relation 




ke role descriptions v;er9 
students did not a^'^^ with 

sole/. J In t\v2 too "..any 
ain rol^G, 

lll'.^i-: o;;t t')? /oh ap_:l LcaLion- . 
r.ts t' ot o'.ev c'^uli Ii-avo 

Lsa^T'i^-rivnt to i^'hich ro^o * 

ERIC 


Given student comments more information should be 
provided on role descriptions. Khile there may have 
been too much reading for the interviev;ing aspects of 
this phace, students need more directions for 
cojnpleting the applications. 
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DATA 
SOURCE 



STUDENT 
TESTS 



STREXT'IS 



Students commented that the following things were what they 
STUDENT liked most atout their role(s): repairing end working with 

QUESTION- radios, working with other people, soldering, and learning 
NAIRES new things (i.e., radio parts). The students felt that 

the tasks were not too complicated or hard for them to do 

(Qij). 



students commented that 
they liked least: readii 
"^ir^ct-Jonr, waiting for 
filling out forms, and 2 



Generally, oil teachers felt the recommended time for 
completing the tasks and level of the tasks to the students 
maturation level was appropriate. All teachers felt the 
flow or integration of one task with another was good. Only 
TEACHER one teacher had minor trouble with the "breaks in flow in the 

QUESTION- activities . Three teachers felt the students had a "high" 
NAIRES understanding of the task directions and/or task materials . 

Generally, the students had no difficulties in implementing 
the tasks. One teacher teamed a slower toy with a more 
mechanically minded student. The students generally enjoyed 
role playing and job activities. The students enjoyed 
meter reading and soldering. In one class, the teacher 
incorporated writing (business letters with the students 
Language Arts cla?s. 



TEACHER 
PATJEL 



In one class, students 1 
tasks due to a high abs< 
blems such as missing p( 
were annoying. Students 
which contributed to the 
tat ion of the task. 



When repairing the radios, the students came in each day 
and set up shop and went right to work. The supply clerk 
role was very valuable. The students had high initial 
interest. Teamwork in the group was very strong. The 
experience was representative of real life situation. The 
letter writing experience was good for the students. All 
roles maintained themselves v:ell up to the end with the 
exception of repair teams who failed to get the radio to 
v/ork* 



Explanation prior to so 
wire coating should be 
work; their trouble def: 
many faulty parts in the 
sufficient information : 
nection of the antenna, 
chart diagram and instrt 
were not numbered. 



OBSERVER 
FORMS 



The observers felt students were generally motivated and 
interested in repairing the radios. The supply man (men) 
understood jobs and functioned very well. The class took 
little time to get started at the beginning of each class 
period and little time for clean-up. The supply clerks 
initiated clean-up in some classes. 



High absenteeism in sort 
problems. The students 
handbook which cause cc 
multimeter. At one sch 
late so at tim^s the cl 
teams were unable to fl 
receiving teacher assis 
know if the radios were 
completed fixing the ra 
result, some students r 
untill all students wer 
Replace'nent parts were 
foreman v;as ."^o busy v;it 
assumed the role of for 
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!• In general, the implementation of the task v;ent 
smoothly. 

2. The teacher panels and the observer rorns v/ere In 
agrcGHient and the supply clerk role was a crucial 
ingredient of the simulation. 

3. The motivation v;ith regard to t\ ^ participation phase 
was high especialliv at the" outset of t:.e pnass. 
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1. There were ucchnica 
errors in ..manuals, 

r. .Ifecial proble.r.s v/i 
:ic.VG been wiLIi conf 
ir.surricient infort 

3 m Jome tea/is complete 
others thus causing 
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FEC0^2EmATT0^]S FOR REVISION 



tat the following tl:inr:s were ^^hat 
iditxg too nany booklets, confusing 
'or .Tiissing parts, '/.Tlting letters, 
id not "beiug able to fix radios. 



s had difficulty implementing the 
bsentee rate. The technical pro- 
parts, errors in company manuals 
nts were unable to repair the radios 
ihe brea!:s in flow in the implernen- 



soldering should explain that ' 

removed. The radios would not 
efied diagnosis. There were too 
the supply kits. There v/as in- 
n in handbook concerning the con- 
a. There vrere flaws in the flow 
tructions. The pages in the iianual 



1. Fe sister banks should be placed in radios as they are 
completed. 

2. At least one radio in good working order should be 
sent with simulation # 

2. Radio parts should be checked for proper working order 

before shipping, 
ii. Additional space for radio parts should be included 

with materials • 



some classes presented management 
its noted several errors in the 
confusions when working v/ith the 
ichool, supplies for the module came 
class had nothing to do. Some 
fix the first radio even after 
listance. They felt they should 
Te repairable. The repair teams 
radios at different times. As a 
; returned to their regulai class 
neve ready for the Summary Phase, 
'c in low supply, "in one class, the 
rith correspondence that the teachc^r 
^oremons^ 



rairlos tliat I not ;,:>rk, ot-* 



rts, 



'n t.:e boo 
j'liirt, etc. 



I../, -.Lrectioni: 
5rr.ation *.n the flo*. 
5te^: repairlnr ra.iios a* ea^ 
Inc. protl^. 
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Irnprove the duality of the radios. Tnsure that 

th<2 radios can be fixed by students. 

"'arefully rcexar.ine dr- .ctions in t:;e Tiaterials 

ar.d tr;o i^low cliart v;ith legard to omissions, 

confusing aspects, and accuracy. 

Tr. zo\Q inrrtancos, make sure enougli spare supplies 

anr; prjrts are available for the simulation. 
^eexz'^ine the Participation ^-hase with regard to 
those students who finish their repair jobs early. 
•\ctiv- ties should be T)ro7ided for early finishers* 
^2 



DATA 
SOURCE 


STPS:iGTH3 


Su-i-navy 


STUDSrW 






TESTS 






STUDEIIT 
•QUESTION- 
NAIRES 





More than half of tne students (53:;) felt t\: 
zo "pull" things t >e:ether at thz cuT'r.iiiation 


TSACP2R 
QUESTION- 
NAIRES 


The students j.n one class complemented 
the v/ork of the supply clerks. 


Throe teachers rated the 3a.a.nary*s effective 
as being "low". They were inconsistent in t 
integration of the nummary with other tasks; 
"poorly" to "well". Three teachers felt tha 
effective" in helping students leern about c 
by others in the simulation. They disagreed 
summary phase in helping students make decis 
other occupational exploration activities, 
was not useful, and two teachers felt it was 
the students couldn't see any point in the a 
tation of their experiences was unnecessary 
knew what the others were doing. They thoug 
grade school "show and tell" experience. Tl 
had some value. One teacher suggested that 
preview to introduce the jobs. 


TEACHER 
PANEL 




Pupils worked so closely together during the 
sharing activity fell so completely flat tha 
D.C.T. use was ineffective in the summary pt 


OBSERVER 
FORMS 


The students indicated liking the video- 
cassette interviews. The more successful 
the repair teams were the more positive 
their attitude toward the module. The 
repair teams commended the work of the 
supply clerks in one class's summary 
activityo 


Little student interest in this phase. Teac 
Students objected about: 1) reading about jc 
were repairable, 3) not ei40ugh freedom to ey 
foreman did not like job since he received t 
about module, 5) The students had difficulty 
actual employment procedures, 6) The student 
paperwork. 






1. The success of the module is depended up 
teams are. While this may b'^ considered 
it is mora than likely a vN'eakness in thi 


TREflDS 


See Tveaknesses colanin 


2a The students, teachers, and observers v/e 
was little student interest in this phas 
Saumary simply v;ere not clear. Nor was 
culminating activity. 

3o Since the students had v/orked so closely 
they felt telling eacr- other v:hat they h 
and too elernentary. 

.;hile the teacher:: •v'i::'we'l t':c '^T/.T. as 
in this activity - p-: que:tionable »'see ^ 
oun^iary "h3et;o 

rtivients objected to f'?e? r.bovt: colu-nn). 
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yTr-ClO^y^ FOR PEVT5TCM 



.:ioo he 



iveness as a culminating activity 

tneir opinions concerning the 
ks; their responses ranged from 
bh3 suinmary phase was "not 
t occupational roles performed 
Bed on the usefuLness of the 
cisions about participation in 

Two teachers felt th3 summary 
was somewhat useful* In general, 
\ summary. They f3lt a presen- 
y since everyone in the module 
ought it was too 'nuch like a 
The introduction to the D«,C.T. 
it it should be used in tho 



ihe simulation that the 
bhat it was abandoned <> The 
phase • 



!• Completely revise the summary approach. 

2. Move t?ie D.C.T. activity to the preview section. 



acher had to prompt responses, 
jobs, 2) not knov/ing if radios 
explore vrork on o'm, the 
i too many student complaints 
Lty relating the experience to 
nts disliked evaluating and 



upon how succecs^'ul repair 
•ei to be a stren'^th by some, 
:is :;iT;Ulation. 
were in ^.Treemont that there 
icse. The objectives of the 
LS it seen as an effective 



ily tc-;:ither *: 



■.•.or 
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1. Tt is clear that this activity was ineffective and 
•^ust be completely revised. 

2. The revisor might consider, for example, field trips to 
a repair shop, guest speakers, having students 
interview repair personnel in the community, etc. 

3. 'tost likely the D.C.T. should be retained for use in 
the .>,imi:?Lation, but placed earlier in the module. 

T usO'-^ earlier, it might be reinforced in the 

. "Itb rerar-- to an.y revisions nale here, the revisor 
•'louJd note that t::e culminating activity of this 

'.t ion *3 the successful :'opair of the radios, 
•.r.'^^.hi nr; b'^yond I'lat '^3.7 soTiewhat seem a'^t'ic] imatic 
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APPENDIX A ; 

Product Services 
Knowledge Test - "What Do You Know?" 
and 

Attitude Scale - "What Do You Like?" 
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The project pre sent ed/reported herein was performed pursuant to a 
grant from the National Institute of Education, Department of Health, 
Education, and Welfare, However, the opinions expressed herein do 
not necessarily reflect the position or policy of the National 
Institute of Education, and no official endorsement by the National 
Institute of Education shoxild be inferred. 

Copyright 197*4^, by the Ohio State University, The Center for 
Vocational Education. 

Copyright for these materials is claimed only during the period of 
development, test, and evaluation, unless authorization is granted 
by the National Institute of Education to claim copyright also on 
the final materials. For information on the status of the copyright 
claim, contact either the copyright proprietor or the National 
Institute of Education. 
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PRODUCT SERVICES COMPANY 



WHAT DO YOU IvNOW? and WIIAT DO YOU LIKE? 



This booklet ?on*^ains two short tests. The purposes of the tests 
are to find out what you know about work in tl»e product services 
field and what kinds of activities you might enjoy doing in product 
services. These tests will not in any way affect your grade. 

Directions : To complete the first test, use the answer sheet and 
pencil that have been provided. In one corner look for the blanks 
marked "Course," "Instructor," etc. Then indicate the. class you 
are in, in the space marked "Course," write in your teacher's 
("Instructor") name, your name, and your school ("Campus") in the 
spaces provided. Then right above where you've been writing, dark- 
en the spaces which indicate your sex and today's date. 

For each question on this test there are several short phrases or 
'Statements listed. Pick the one that best describes your answer 
and then darken the appropriate space opposite the item number on 
whe answer sheet. Note: on the answer sheet the item numbers go 
across the page instead of up and dov/n. 

If you don't know the answer to a question, GUESS. 

Thanks for your help. 



You may turn the page and start as soon as you have completed 
reading the above paragraphs. 
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PRODUCT SERVICES COMPANY 



If 



WHAT DO YOU KNOW? 



II 



FILL IN THE FOLLOWING INFORMATION 



Name 



Age 



Grade 



STAKT THE TEST 

1. Tho job of foreman in a product services company is most 
similar to the job of 

a. Welder 

b. Machinist 

c. School principal 

d. Physician 

2. Which of the following functions is not_ a part of the product 
services field? 

a. Manufacturing products 

b. Maintaining products 
Altorinc; products 

d. Installing products 

e. Repairing products 

i. Four members of the insect world wandered into the personnel 
office of Ace Product Services, Inc. As personnel director, 
pick the iusoct most qualified to work in your supply depart- 
ment, 

a. Daisy Dr-.-jouriy 

b . Paul Praying ilanlis 

c. Terry Tick 

d. Annabelle Ant 

4. A product war nty is most like 

a. An 1*0. U. 

b. A bill of sale 

c. A promise 

d. A bill of lading 
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5. 



"Bench worK" refers to 



a. Repairs done at the customer *s home 

b. Repairs done in the factory 

c. Repairs done at a service center 

d. Both a and c 

e. Doth b and c 



6. As a worker in the product services field, you might be 
expected to perform \hich of the following services? 

a. Install a oroluct to make it ready for use 

b. Repair motive ^^roducts 

c. Alter ^ product to meet the needs of the customer 
d . All of i \e above 



7. What job in the volunteer navy would probably prepare you to 
work in the supply division of a product services company? 

a* Pharmacist's mate 

b* Shore patrolman 

c. Typing pool specialist 

d* Engine maintenance specialist 

8. When products malfunctioned aboard the recent Apollo space- 
shot, the astronauts used ground contacts in what way? 

a. As a group type of foreman 

b. As problem analyzers 

c. As supply personnel 
■U Only a and b 

9. How are costs for servicing products generally determined? 



a* By negotiations between the customer and the product 
services company 

By tables of standard fees for parts and labor 

c. By estimates made at the time the repair job is first 
brought in 

d. By a and i ^qot'aer 



10. Product service people »ise information in diagrams made by 
whom at the manufacturing company? 



a. Research and development personnel 

b. Control personnel 

c. Technical service personnel 

d. Administrative personnel 
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11. To adequately service a product, the manufacturer should supply 
the product services company with 

a» Diagrams of tho detailed product design 

b. All the tools necessary to service the product 

c. Costs for labor services 

d. Instructions for making alterations in the product 

12* What kind of work uone by repair personnel is similar to the 
work done in P'^oduct- manufacturing? 

a. Designi ^.j products 

b. Fabricatir*^' products 

c. Inspecting the quality of products 

d. Distributing products 

13. Servicing a television set in the repair shop might be referred 
to as 

a. Electrical, bench service 

b. Electrical, mobile service 

c. Mechanical, bench service 

d. Mechanical, mobile service 

14* Product service is an extension of manufacturing and 

a. Marketing 

b. Maintenance 
c» Construction 

d. All of the above 

15. Repair personnel should keep records of 

a. Time spent repairing a product 

b. Parts used in repairing a product 

c. The inspection and testing of a product 

d. All of thp above 
e» Only a and b 

16 ♦ The first st. . • servicing a product is 

a. Locating the problem (s) in the product 

b» Labelling the product with the owner's name 

c. Cleaniny the product 

d. Repairing the product (including parts replacement) 
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17 • Frequently, customers write 1:0 product services companies to 
complain about repair work, to ask about supplies, or to 
ask general questions about products. Who hat5 responsibility 
for responding to their letters? 

a. Repairmen 

b . Foremen 

c. Supply perscnnel 

d. Each of the above, depending upon the kind of question the 
customer asRs 

18. The comic stiip character Dick Tracy would probably do best in 
which of t-40 lollowing activities in the product services 
fields? 

a. Preparia<i cost reports 

b. Identifying problems 

c. Evaluating repair work 
Supplying parts 

19. The duties of a repairman are to 

a. Examine, test, and repair products 

b. Examine and repair products, and assign costs to the 
repair job 

c. Test and repair products, and assign costs to the repair 
job 

d. Examine and repair products and check the final repairs 

20. A well-trdined T.V. repairiiian should be able to 

a. Identify T.V. parts on sight 

b. Estimate the cost of a repair job while in the customer's 
liomo 

c. Determine the amount of time necessary to repair the T.V. 
after a simple inspectic of it 

d. All of the above 

e. Only a and c 

21. The cost of repairing a new product under warranty is paid by 

a. The product buy^r 

b. The product services company 

c. The product seller 

d. The product manufacturer 

22. As the product manager of a product services company you must 
determine the cost of servicing a product. What should you 
consider to determine the cost? 

a. The amount of time necessary to repair the product 

b. The price of the part of the product that had tlo be replaced 

c. The amount of cleaning of the product that had to be done 

d. All of the above 
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23. 



Allen Apple wants to be a product repairman when he finishes 
high school. Which of the following courses that he is presently 
taking will be of most benefit to him? 



a. English 

b* Mechanical drawing 

c. Industrial arts 

d. All of the above 

e. Only b and c 

24. It is the job oC supply department personnel in product services 
to 

a. Take \*iv» itory of too'ls and parts 

b. Assign radios needing repair to repairmen 

c. Keep records of how long it takes to repair products 

d. Complete warranty cards for repaired products 

25. To determine costs for servicing a product tne product repair- 
man must keep track of 



a. Time required for servicing 

b. Replacement parts needed 

c. Alterations or changes made in the product 

d. All of the above 

26. Who, in a product service company, makes the decisions about 
honoring warranties? 

a. Repairmen 

b. Foremen 

c. Supply personnel 

d. Manufacturer 

27. Which of the following duties is the responsibility of supply 
personnel in a product services company? 

a. Determininf^ parts needed for repairing a product 

b. Completing shipping listsv'for parts 

c. Determin.oq '>£t3 for repairing a product based on parts 
used 

d. Keeping track of wi4rit parts were used to repair a product 

28. A service repair bill is figured by adding 

a. The wholesale pric^- of parts to the estimated r<?^air price 

b. The wholesale price of parts to the total labor price 

c. The retail price of parts to the total labor price 

d. The retail price of parts to the estimated repair price 
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29. "Trouble-shoot incf" is a process used in oervicing damaged or 
defective products. From the following statements pick the 
one that best describes trouble-shooting. 

a. Trouble-5^hooting is mainly guesswork, requiring limited 
knowledge of product design 

b. Trouble-shooting is mainly guesswork, requiring a great 
deal of knowledge of product design 

c. Trouble-shoot infj is logical, step-by-step procedure 
requirinq limited knowledge of product design 

d. Troubl, -si;;;ot,ing is logical, step-by -step procedure, 
requirincj great deal of knowledge of product design 

30. The activities of a certain member of the Animal Kingdom would 
perhaps qualify him to work in the repair area of a product 
services company- Pick the animal. 

a. Squirrel 

b. Pack rat 

c. Beaver 

d. Raccoon 

31. Otto Sight is an outgoing person who has spent three years 
in the service doing repair work. What type of product 
service v/ork would best suit him in civilian life? 

a. Working in a product services company repairing products 

b. Intalling and maintaining products in customers' homes 

c. Keeping track of supplies for a product services company 

d. Making diagrams of products for use in repair work 

32. As a ^foreman of a product services company you v/ill be 
responsible for 

a. Assigning jobs to workers 

b. Supervi:^,incj the work of other employees 

c. Organizing the work of repairmen 

d. Maintaining shop cleanliness 
G. All of tiiC ovc 

33. Which of the following processes are most important for supply 
personnel in the product services field to know? 

y 

a. Steps necessary to repair products 

b. Product design 

c. Methods of record keeping and inventorying 

d. Trouble-shooting techniques 
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PRODUCT SERVICES COMPANY 



"WfiAT DO YOU LIKE?" 



This is the s^ror.d set of questions for you to answer. The 
purpose of tiieso questions is to find out what types of activ- 
ities you might enjoy doing in the product services field. We 
would also like to know what reasons you have for liking these 
activities. 

There are only seven (7) questions to answer. Directions for 
answering are found on each page. Write your answers directly 
on the page. 

After you have completed the questions, please return this book- 
let and you answer sheet from the first test to your teacher. 
Thanks for your help. 



Please turn the page and begin the questions as soon as you have 
finished reading the above paragraphs. 
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7. Delow is a conversation between two people, i'ciirou 2 in 
looking for a job and is considering work in the product 
services field. Person 2, a worker in protluct services, 
is thinking about giving person 1 some advice. Pretend 
that you arc person 2, giving advice. Simply complete 
person two's advice at the end of the conversation* 



Person 1: lii pal, how's it going? 

Person 2: Well, aside from having my car stall in the 
morning rush hour, everythings pretty good. 
How's it with you? 

Person 1: r'ine, but I've been thinking about going into 
a different line of work. You know I'm handy 
with tools and I do enjoy repairing things. 
Don't you work for a product repair firm? 

Person 2: Yes, I've been with Ace Product Services, Inc.# 
for the last two years. 

Person 1: Listen, would you help me out? Would you tell 
me what kind of experiences or activities might 
help lac to prepare for a job in the product 
services field? 

Person 2: Sure, here's what I would do if I were you. 
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APFEHDIX B ; 

Product Services 
Student Questionnaire - "What Do You Think?" 



4-. 
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The project presented/reported herein iras performed pur<!uaixt to a 
great from the National Institute of Education, Department of Health, 
Education, and Welfare. Hovever, the opinions expressed herein do 
not necessarily reflect the position or policy of the National 
Institute of Education, and no official endorseBent by the National 
Institute of Education should be inferred. 

Copyright 197U, by the Ohio State tiniversity. The Center for 
Vocational Education* 

Copyright for these materials is claimed only during the period of 
development, test, and evaluation, unless authorization is granted 
by the National Institute of Education to claim copyright also on 
the final materials. For information on the status of the copyright 
claim, contact either the copyright proprietor or the National 
Institute of Education. 



BEST COPY RVMUBU 



89 



WHAT DO YOU THINK?" 



Now that you have completed this simulation, the people who developed 
it would like to find out what you think about your experience. Your 
ideas will help to make the simulation better. Remember # THIS IS NOT 
A TEST and your answers wil"* not be graded. So feel free to check 
and to say what you think about this simulation. 

To complete the questionnaire first fill in the information requested 
below. 

FILL IN THE FOLLOWING INFORMATION 

Name ... D^^® — 

School City 



Age 

Grade (circle one) 8th 9t:i Other (please specify) 

Sex (circle one) Male Female 

Subject taught in this class [ 

Teacher's name^ ^ ^ ^ 

START THE QUESTIONS 

This is a list of statements which describe ideas about the simulation 
module you have just completed. Answer each statement b^^ checking the 
category which comes closest to what you think: 

Check " AGREE " if you think the statement is true for you. 

Check " DISAGREE " if you think the statement is NOT true 
for you* 



AGREE DISAGREE 



1. The preview and the other activities at the 
beginning helped to prepare me for the 
simulation. 

2. The role descriptions gave me little 
information helpful in choosing a role. 

O 
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AGREE DISAGREE 

3. I selected a role by myself. 

4. The teacher helped the class to 

select roles. 

5. Some of the tasks were too complicated 

or too hard for me to do. 

6. The summary helped me to "pull things 

together . " 

7. The simulation preview, activities and 

summary fit well together. 

8. There were too many forms to fill out with 

this simulation. 

9. The directions in the materials were clear 

to me. . 

10. The teacher explained a lot of words. 

11. The pretest and posttest were difficult for 

me. 

12. The booklets and resource materials were 

easy to read. 

13. The teacher explained a lot of ideas. ___ 

14. The simulation was too short. 

15. Sometimes I had nothing to do. 

16. Sometimes I had too many things to do in 

^"his role. 

17. I learned quite a bit about jobs in this 

field of v^rk. 

18. I learned very little about how to work 

with other people. . 

19. The simulation did not help to answer 

some of the questions I have about jobs. 

20. I enjoyed working with other students 

during the simulation. 
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Answer these questions by circling the letter in front of the phrase 
that b^st describes your answer. 



21. How much do you feel you learned about jobs in this field of work 
from the simulation? 

a. Very b. Much c. An average d. Little e. Very 
much amount Little 

22. Kow much trouble do you feel you had knowing what to do next in 
the simulation? 

a. Very b. Much c. An average d. .Little e. Very 
much amount little 

23. How would you judge the length of time you spent participating 
in this simulation module? 

a. Too b. Long c. Just d. Short e. Too 
long right short 

For the next questions f write in your answers. Space has also been 
provided for you to write in any comments/suggestions you might have. 
You are encouraged to do so • 

24. What role (or roles) did you play in this simulation? 



25. Name some of the things you liked most about the role(s) and some 
of the things you liked least about the role(s) • 

Liked Most Liked Least 



26. What other roles in the simulation did you find interesting? 



27. Why did you find this role (or roles) interesting? If you did 
not find any other roles interesting # can you say why? 



9Z 



28. Name some o£ the materials (Examples: slides, tapes, films, 
resource materials, booklets, etc.) you liked most and some 
of the materials you liked least . If you did not use any 
materials, check this space. 

Liked Most Liked Least 



29. Compared to your former feelings, how do you now feel about 
jobs in this area of work? 

WHY? 

I am more interested now 

I am less interested now 



I was not interested and 

I feel the same way now 

I was interested and I 

feel the same way now 

30. Did you discover any new interests by participating in this 
simulation? 

Yes, I am now interested in 

No 



31. Name some of the things you liked most about the simulation 
and some of the things you liked least about the simulation. 

Liked Most Liked Least 
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32. Write down some of your ideas on how the simulation might be 
made better. 



As soon as you have completed these questions # turn in this booklet 
to your teacher. 

Thank you. > 

I 



ERLC 

■~ 94 



and 

O^Ditral Module Svalucbioa 




Kami qtacsTiomiAiRB 



BEST CdPY AVAILABLE 



The project presented/reported herein was performed pursuant to 
a grant from tlu^ National Institute of Education, Department of 
Health, Education, and Welfare. However, the opinions expressed 
herein do not necessarily reflect the position or policy of the 
National Institute of Education, and no official endorsement by 
the National Institute of Education should be inferred. 

Copyright 1974 by the Ohio State University, The Center for 
Vocational and Technical Education. 

Copyright for these materials is claimed only during the period 
of development, test, and evaluation^ unless authorization is 
granted by the National Institute of Education to claim copyright 
also on the final materials* For information on the status of 
the copyright claim, contact either the copyright proprietor or 
U.a National Institute of Education. 
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MIDWAY MODULE 0^'ESTIONNAIRE 

The questionnaire is divided into several sections. Each section 
in order corresponds to a part or a phase of the simulation module. 
The last sections deal with your overall perceptions at this point 
in time regarding what has happened in the module* 

Fill in the information requested at the top of the questions. 
Then answer each question by circling the letter in front of the 
phrase that best describes your answer, unless given other specific 
directions in the question. Space has also been provided for you 
to write in anv ccmiaents/suggestions you might have. You are en-- 
couraged to do so. 



FILL IM THE FOLLOWING INFORMATION 

Teacher Name School 



Date ^Part of the Module you are now working on 



INTRODUCTION TO SIKULATION 



I. Overall^ hov would you rate the technical quality (appearance, 
ease of use^ etc.) of the slides and booklet? (Answer both 
parts of question if applicable.) 

Comments 





Slides 




Booklet 


a. 


Very Good 


a. 


Very Good 




Good 


b. 


Good 




Average 


c* 


Average 


d. 


Poor 


d. 


Poor 


e. 


Very Poor 


e« 


Very Poor 



2. In what order v/ould you recommend the use of slides and the 
booklet? (Chorj-^e only one). 

a. Use both in any order 

b. Use both with booklet first 
c* Use both with slides first 

d. Use the booklet only 

e. Use. the slides only 

f . None of the above 

3. Please record any strengths and/or weaknesses that you observed 
while working with this part of the simulation module_ 
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MODULE PREVIEW 

4* Indicate the form of presentation used (e.g., booklet, sound- 
slide, g5«ne, etc.) 



5. llow would you rate the technical quality (ease of use, appear- 
ance, etc.) for media and/or the illustrations for booklets?' 

a. Very b. High c. Medium d. Low e. Very 
High 

6. In your judgment, did this form provide pertinent information 
that students could use in making decisions about module parti- 
cipation? 

a. Very b. Rather c. Average d. Not very e. Not Perti- 

Pertinent Pertinent Pertinent nent at all 

7- Overall, how would you rate the ability of the "Preview" form 
for motivating students to participate in the module? 

a. Very b. High c. Medium d. Low e. Very 
"ioh LOW 

8. Plbase record any strengths and/or weaknesses that you observed 
While working with this part of the simulation module 



IKEKM IA TION PHASE/ROLE SELECTION 

9* Indicate the form of presentation (e.g., slide-tapes, booklets, 
etc.) used in the Preparation Phase. 

10. How would you rate the technical quality (e.g., ease of use, 
appearance, etc.) for media and/or illustrations for booklets? 

a. Very b. High c. Medium d. Low e. Very 
High Lqv^ 

11. How well did the Preparation Phase fit together with the Module 
Preview? (i.e., did the Preview flow into the Preparation Phase?) 

a. Very b. Well c. Somewhat d. Poorly e. Very 
Well Poorly 

12. Did the initial role descriptions provide students with enough 
information for selecting roles? 

a. Yes, the information was very adequate 

b. Yes, the information was rather adequate 

c. No, the information was rather inadequate 

d. No, the information was very inadequate 
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13* If schematic devices (e.g. schedule cards) were available to 
help select roles, did students understand how to use them? 

a. Yes, with little or no help 

b. Yes, with some help 

c. Yes, with a great deal of help 

d. No 

e. Not applicable 

14. Were the students able to independently select themselves into 
roles? 

a. Yes, .7ith Ixctle difficulty 

b. Yes, with some difficulty 

c. No, some teacher assistance was necessary 

d. No, extensive teacher assistance was necessary 

15. If you had to help students select roles, please describe the 
nature of that assistance (e.g. asked students to draw lots when * 
several wanted the scune role; explained use of schematic device, 
etc.) in the space below: 



IG. Please record any strengths and/or weaknesses that you observed 
while working with this part of the simulation module 



FIRST TASKS 



This section includes questions about the implementation of tasks, 
the flow of one t-ask to another, etc. We would like your reactions 
to the tasks up t.^ this point. We realize that you have not completed 
all of the tasks. We will ask you about the later tasks in the short 
questionnaire administered after the module has been completed. 

17. In general, was the recommended time appropriate for completing 
the tasks? 

a. Yes 

b . Somewhat 

c . No 

If "No/\ please specify the task(s)_ 
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18. In general, were the tasks appropriate to the maturational 
level of the students? 

a. Yes 

b. Somewhat 

c. No 

If "No," please specify the task{s)^ 



19. How would you rate the flow or integration of one task with 
another? 

a. Very b. Good c. Average d. Poor e. Very 

Good Pqqj. 

20. Did you hav- any special problems or any particular breaks in 
flow? 

a. Yes 

b. No 

If "Yes," please specify 

21. How would you rate student understanding of task directions 
and/or task materials? 

a. Very b. High c. Average d. Low e. Very 
High ^ 

If "Low," or "Very Low," please specify 



-l2. Did the students have any major problems in implementing the 
tasks? 

a. Yes 

b. Somewhat 

c. No 

If "Yes," please specify 



23. Please record any strengths and/or weaknesses that you observed 
while worki--- ..'i^h this part of the simulation module 



STUDENT INTEREST AND UNDERSTANDING 



24. In general, were the directions in the module clear enough for 
students to understand what was expected of them? 

a. Very b. Clear c» Average d. Unclear e. Very 

Clear Unclear 
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25. In general, was the vocabulary of the module consistent with 
the maturational level of the students in the simulation? 

a. Yes, most b* Yes, some c. No, not d. No, none 
of it of it much of it of it 

26. In general, were the students able to understand the concepts 
presented in the materials? 

a. Yes, most b. Yes, some of c. No, not much d. No, not 

of the time the time of the time at all 

27. In genexfai, did the materials stimulate student interest? 

a. Yes, moot b. Yes, some of c. No, not much d. No, not 
of the time the time of the time at all 

28. Did your students experience problems with the reading level 
of this simulation module? 

a. Yes, many b. Yes, some c. Yes, but few d. No 

problems problems problems problems 

29. While working with the students in the simulation module, did 
you spend extra time in reviewing the basic concepts presented 
in that phase? 

a. Yes, I spent much time 

b. Yes, I spent little time 

c. No, I didn*t spend any time 

30. Please record any strengths and/or weaknesses thav you observed 
while working with this part of the simulation module 



ADEQUACY OF MATERIALS - OVERALL PERCEPTIONS 



31. In general, how well did the transitions from phase to phase 
of the module proceed? 

a. Very b. Well c* About d. Poorly e. Very 
Well Average Poorly 

32. Up to this point, are there any additions, deletions, or 
changes in the module that you feel should be made? 

a. Yes, make the following changes 



• b. No changes are necessary 
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33. Are there any parts of the module that "just didn't work?" 
a. Yes, the following parts_ 



b. Uo, all parts worked well 

34. All factors considered ^ which specific set of materials would 
you rate as the best? 



35, All factors considered ^ which specific set of materials would 
you rate as the worst? 



36. Up to this point f add as many comments and/or suggestions for 
revision of the module as you might have. 
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GENERAL MODULE 
EVALUATION 
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The project preeented/reported herein perforaad pursuant to a 
grant trm the National Institute of Education, Departnent of Health, 
Education, and Welfare* Hovever, the opinions expressed herein do 
not necessarily reflect the position or policy of the Eaticnml 
Institute of Education, and no official endorssMnt tqr the Rational 
Institute of Education should he inferred* 

Copyright 197U, hy the Ohio State University, The Center for 
Vocational Education* 

Cqporright for these materials is claimed on3gr during the period of 
derelqpnent, test, and evaluation, unless authorisation is granted 
by the Istional Institute of Education to claim cppyri^t also on 
the fioal materials* For inftenation on the status of the copyright 
claim, contact either the copyright proprietor or the National 
Institute of Education* 
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GENERAL MODULE EVALUATION 



This questionnaire is divided into several sections. The 
first two sections correspond to the last tasks in the module 
(i.e., those from the Midway Questionnaire to the end of the 
module) and to the Summary Phase. The last sections deal with 
general teacher and student background and your overall percep- 
tions of the quality of the materials, implementational problems, 
student interest and understanding, etc. 

Answer each question by circling the letter in front of the 
phrase that best describes your answer, unless given other specific 
directions in the question. Space has also been provided for you 
to write in any comments/suggestions you might have. You are en- 
couraged to do so. 
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GENERAL MODULE EVALUATION 



FILL IN THE FOLLOWING INFORMATION 



Teacher Name 



School 



Sex 



Years of Teaching Experience 



City 



LAST TASKS 

1. In general, was the recommended time appropriate for completing 
the tasks? 

a. Yes ^ 

b. Somewhat 

c. No 

If "No#" please specify the task(s)_ 

2. In general r were the tasks appropriate to the maturational level 
of the students? 

a. Yes 

b. Somewhat 

c. No 

If "NO," please specify the task(s)__ ^ 

3. How would you rate the flow or integration of the tasks with each 
other? 

a. Very b. Good c. Average d. Poor e. Very 
Good Poor 

4. Did you have any particular breaks in flow? 

a. Yes 

b. NO 

If "Yes," please specify 

5. How would you rate student understanding of task directions and/or 
task materials? 

a. Very b. High c. Medium d. Low e. Very 

High Low 
If "Low," or "Very Low," please specif y 

6. Did the students have any major problems in implementing the tasks? 

a. Yes 

b. Somewhat 

c. No 

If "Yes," please specify 
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1. Please record any strengths and/or weaknesses you observed while 
working on this part of the simulation module: 



SUMMARY PHASE 

8* How would you rate the effectiveness of the Summary Phase in pro- 
viding a reasonable culmination, i*e«# in tying together concepts, 
roles # etc. presented in the module # to the simulation experience? 

a. Very b. High c* Meditim d. Low e* Very 
High Low 

9. To what extent was the Summary Phase integrated with the immedi*- 
ately preceding activities or tasks? 

a. Very b* Well c. Average ^d* Poorly e* Very 
Well Poorly 

10 • How would you rate the effectiveness of the Summary Phase in 
helping students learn about occupational roles performed by 
others in the simulation? — 



a. Very b« Somewhat c* Not 

Effective Effective Effective 

11. How useful do you feel the Summary Phase would be in helping 
students to make decisions about participation In other occupa- 
tional exploration activities # i.e., other simulation modules, 
etc • ? 

a« Very b* Somewhat c« Not 

useful Useful Useful 

12. Please record any strengths and/or weaknesses you observed while 
working on this part of the module: 
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TEACHER BACKGROUND 



13. In what kind of group setting (e.g., English classroom, math 
classroom, students from study hall, students from a guidance 
group, etc.) and at what grade level did you introduce this 
simulation? 

a. Group Setting (please specify) 

b. Grade Level (please specify) 



14. Have you had any previous experience with simulation as an 
instructional technique? 

a. Yes, as a teacher 

b. Yes, as an observer 

c. Yes, as a participant 

d. No 

15. If you answered yes to question 14, briefly describe the 
nature and extent of your previous experiences with simulation 
If your response to question 14 was ••No*', please proceed to 
question 16. 

a. My previous experiences with simulation include 



16. Which of the following statements best describes your reasons 
for participating in the pilot test of this simulation module? 

a. Wanted to try out new ways of organizing instruction for 
students 

b. Have an interest in Career Education 

c. Thought material was of value for students 

d. Have a general interest or curiosity 

e. I was requested to participate 

f . Other, or some combination of the above (please specify) 
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STUDENT BACKGROUND 



17. How were students selected to participate in the simulation? 

a. Students volunteered from the class 

b. The class, rather than the students, volunteered 

c. Student volunteers from a study hall 

d. Other, please specify 



18. If you had volunteer students participating in the simulation, 
which of the following reasons best describes your perception 
of why they participated? If you did not have any volunteer 
students, please proceed to question 19. 

a. Interest in trying something new 

b. Interest in particular area simulated 

c. Interest in careers 

d. Interest in just getting out of class or study hall 

e. Other, or some combination of the above (please specify) 



f. I can't really guess at the reason (s) 

19. Indicate any special characteristics of this class, e.g., many 
slow readers in class; many students with exceptionally good 
verbal skills; etc., which may bias the results of the pilot 
test of this module. Also, describe how you feel the results 
will be biased by these characteristics. 

a. Characteristics Biases Produced 



b. No special characteristics 



IMPLEMENTATION ^OF THE MODULE 

20. How well did the in-service training prepare you to work with 
the module? 

a. Very b. Well c. Somewhat d. Poorly e. Very 
Well Poorly 

21. Did the in-service training provide you with a general under- 
standing of your role in the module implementation? 

a. Yes 

b • Somewhat 

c. No 

If "NOy" please specify 



22. While working with this module # did you have to allot (or spend) 
more time than you nojnnally would for preparation (exclude the 
time spent in in-service training)? 

a. Yes, specify additional time in hours 

b. Some extra time was necessary 

c. No extra time was necessary 

23. How sizable was the job of managing/coordinating (helping 
students, keeping track of materials) this simulation module 
for you? 

a. Very b. About c. Not 

Sizable Average Sizable 



ADEQUACY OF EVALUATION MATERIALS 



24. Do you feel that the knowledge (Wlxat do you know?) and the atti- 
tude (What do you like?) tests were adequate measures of the 
material contained in the module? (Answer both parts of the 
question.) 

Knowledge Test Conmients Attitude Test Comments 

a. Yes a. Yes 

b. Somewhat b. Somewhat 

c. No c. No 

25. To what extent was the knowledge test difficult for students? 

a. Very b. Difficult c. About d. Easy e. Very 
Difficult Average Easy 
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STUDENT UNDERSTANDING, INTEREST, AND PARTICIPATION 



26. In general, were the directions in the module clear enough for 
students to understand what was expected of them? 

a. Very b. Clear c* Average d* Unclear e. Very 

Clear Unclear 

27. In general, was the vocabulary consistent with the maturational 
level of the students in the simulation? 

a. Yes, most b. Yes, some c. No, not much d. No, none 
of it of it of it of it 

28* Did your students experience problems with the reading level of 
this module? 

a. Yes, many b. Yes, some c. Yes, but few d. No 

problems problems problems problems 

29. To what extent do you feel students were receptive (interested 
in, excited by' to simulation as a way of learning? 

a. Very b. Receptive c. Average d. Non- e. Very non- 
Receptive Receptive Receptive 

30. To what extent do you feel that students were receptive (interested 
in, excited by) to the content of this particular module? 

a. Very b. Receptive c. Average d. Non- e. Very non- 
Receptive Receptive Receptive 

31. Was there any change in student interest or motivation as they 
progressed through the module? 

a. Yes 

b* Somewhat 

c. No 

If "Yes," interest changed as follows . 



32. Do you feel that this module reinforced or helped to build the 
student's ability to make decisions? 

a. Yes 

b. Somewhat 

c. No 

d. Don't know 

If "Yes," please specify how 



33. In your judgment, how much did the students learn about the 
process of simulation (role playing, problem solving, group 
interaction , etc . ) 

a. Very b. Much c. An average d. Little e. Very 
Much amount Little 

34. In your judgment, how much did students learn about the content 
of the module? 

a. Very b. Much c. An average d. Little e. Very 

Much amount Little 

35. Are there any students or groups of students (e.g., some students 
may have difficulty working in small self -directed groups) that 
you feel would have difficulty in participating in simulated types 
of experiences? 

a. Yes 

b. No 

If "Yes," please specif y_ 



36. For what grades would you consider this module to be appropriate? 

a. 10th or b. 9th c. 8th d. 7th or e. Other 
higher lower 

37. Ideally, how many students should participate in this module? 
Number of students; 



38. In general, did this module change the working relationships 

(personal interactions) between you and participating students? 

a. Yes 

b. Somewhat 

c. No 

If "Yes," or "Somewhat," the relationship changed as follows 
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OVERALL PERCEPTIONS AND RECOMMENDATIONS 



39. Overall, how would you rate the quality of the module? 

a. Very b. Good c. Average d. Poor e. Very 
Good Poor 

40. If possible, would you use this module with students again? 

a. Yes, with no modifications 

b. Yes, with minor modifications 

c. Yes, with major modifications 

d. No 

Please comment, if you wish 



41. Would you recommend this module to other teachers? 

a. Yes 

b. No 

Please give your reason (s) 



42. Were the main ideas and themes presented with logical consistency 
in the content of the module? 

a . Yes 

b . Somewhat 

c. No 

If "No," please specify where the problems occurred^ 



43. All factors considered, which specific set of materials would 
you rate as the best? 



44. All factors considered, which specific set of materials would 
you rate as the worst? 



45. Add as many comments and/or suggestions for revision of the 
module as you might have. 
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SIMULATION OBSERVERS FORM ~ A 



This instrument is designed to obtain samples of on-going classroom 
behavior of students using simulation modules. These modules are being pilot 
tested as a part of the Occupational Kxploration Program by the Center'^for 
Vocational and Technical Education at The Ohio State' University and the Jefferson 
County Public Schools. 

The observation form is made up of a set of three sheets. Each set contains 
four parts: the heading, media section, general comments and the interaction and 
activities section. An observation form set is to be used for each period that 
is observed. The parts of each set are discussed below. 



The Headin|3^ 

The heading simply identifies the time, place, observer and the portion of 
the module that was observed. For ease of completion, the observer's name, school, 
and module have been given a number code. Simply circle the appropriate number 
according to the code below: 



Observer: 
School: 



Numbers will be assigned 



1. 
2. 
?. 



Alameda Junior High 
Hamilton Junior High 
Lake Junior ijigh 
Wheat Ridge Junior High 



Module: 1. Communications 

2. Product Services 

3. Insurance 
Health & Welfare 



Date : 

Activity or 
Activities : 



Indicate the date of the observation 



Indicate either the title of the activity i.e. "Preview'' 
''Summary*' or the number i.e. "Task 3" etc. Several 
spaces are provided in the event that more than one task 
or activity takes place in one period. 



1. Media 

The media section has two spaces that should be completed each time the 
pupils use some form of media. In the space following the type of media used, 
place a check (i^) ^^ach time the media is used. For each (/^, the number of 
students using that form of media should be indicated in the No. of Students 
Column. (See sample) ♦ 
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2. General Comments 



The general comments section is designed to capture comments that do not 
lend themselves to the other categories . Two categories that are of continuing 
interest is the amount of time spent by pupils getting ready to start and the 
amount of time cleaning up and getting ready to leave. You will note that these 
categories are pre^printed on the observation for^a. (Examples of general comments 
of interest appear on the sample form). 



3» Interaction & Activities 

This section is designed to provide several kinds of information: 

a. Ffow frequently do certain categories of events occur? 

b. What size group were the students in during the event? 

c. What were the circumstances surrounding the event? 
and in some instances: 

d. How long did the event last? 



The procedure for this section is as follows: Each time one of the events 
in either the student or teacher activity columns occurs. record an arabic number 
in either the total group or sub-group column. (The total group column is 
appropriate when all of the students are working together). (The small group 
colum is appropriate when the students are working individually or in two or 
more Rroups). Begin with number 1 each period; then number the events consecutively 
throughout the period. The comment section is provided in order that a very 
brief comment or ke^*- word may be used to explain each arabic number. (See example). 
NOTE ; The events for the entire period should be numbered consecutive].y even 
though they are sciittered between categories a through f. This system will allow 
the evaluation s.taff to reconstruct what happened during each period. 

If a number of questions about the same thing occur in category a, the 
numbers may be bracketed as is shown in the sample. Also if a number of questions 
follow each other, it is of interest how long the questioning took. (Again see 
the excunple ) . 
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Explanation of Sample Form 



?teading> This form was completed by observer number 2 at Alameda Junior 
High on Task 1 of the Communicatiors Module, March 21, 197^*. Eight pupils were 
present the day of the observation. 



Media Section. 



During the observation period, the students used two media forms in Tasks 
1 & 2. They began with the so*ind/slide presentation, switched to the booklet 
and finally used the booklet as they began Task 2. The media In each instance 
was used by the total group. 

General Comments . 

Some of the general comments relate to other parts of the observation form 
in the sample, others are simply given as examples of the kinds of comments that 
might be appropriate. Note that it took the students 5 minutes to get started 
and 3 minutes to get ready to leave. 

The comment space is designed to capture your overall impressions of special 
or noteworthy' events occurring during the period. 

Interaction and Activities Section . 

This section provides a sequential history of what happened during the 
period. By reading the Ai^abic numbers and comments in order, the sample allows 
the following reconstruction of events; 

1. The pupils began as intended by viewing the slide tape as a 
total group. 

2. Someone asked for help with the slide tape machine. 

3. As the teacher helped with the machine, other students began 
to "horse around". 

^. The teacher, discovering the machine was broken, directed the 
pupils to use the booklet instead* 

5. Teacher stopped the horseplay and redirected the actions of 
the miscreants. 

6. A pupil asked for help in finding a booklet. 

7. A pupil did not understand the booklet. 
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8, 9i 10, 11. A number of questicia were asked regarding what 
should be done fo3 mowing the booklet - 5 minutes 
were consumed. 

12. The pupils broke up into eproups at this point. (The observer 
is now fccusing on one of the groups only). 

13. The. teachar redirected the leader to his proper group. 

ll*. The small group assembled & began to discuss their task as 
intended. 

15. The task was completed, the product (a report in this instance) 
was completed. The total group moved on to Task 2 as the time 
came to begin the cleanup/put-away procedure. 



Footnotes 

Obviously all that transpired during the period was not recorded. No 
observer should feel they must capture every siHijle event or question. With 
experience and through use of the flow chart for the module being observed, 
observers will become increasingly capable of capturing the more significant 
questions, events, .etc. 

Should questions arise, do not hesitate to contact John Hadloff, Jeffco 
Career Education Office - I423-70IO. 
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